Evaluating research - Peer review team assessment and journal-based bibliographic measures: New Zealand PBRF research output scores in 2006

This paper concerns the relationship between the assessment of the research of individual academics by peer or expert review teams with a variety of bibliometric schemes based on journal quality weights. Specifically, for a common group of economists from New Zealand departments of economics the relationship between Performance-Based Research Fund (PBRF) Research Output measures for those submitting new research portfolios in 2006 are compared with evaluations of journal based research over the 2000-2005 assessment period. This comparison identifies the journal weighting schemes that appear most similar to PBRF peer evaluations. The paper provides an indication of the ‘power or aggressiveness’ of PBRF evaluations in terms of the weighting given to quality. The implied views of PBRF peer review teams are also useful in assessing common assumptions made in evaluating journal based research

The performance of New Zealand universities in international rankings

Summary: Despite being subject to much criticism, international university rankings are attracting more coverage, are proliferating, and appear to be here to stay. Most countries and universities at the very least monitor the results of the rankings when they are published. Many universities strive to improve their rankings. In this study, we examine the performance of New Zealand universities in the ‘big three’ university rankings: the Quacquarelli Symonds (QS) World University Rankings, the Times Higher Education (THE) World University Rankings, and the Academic Ranking of World Universities (ARWU). In the high-profile QS ranking, there has been a downward trend in the rankings for the top-placed New Zealand universities. However, all of our universities are currently placed in the QS top 500, something not achieved by the Australian, Canadian or United Kingdom university systems. Also, the performance of New Zealand universities in the QS subject-level rankings tend to be higher than in other rankings. In the THE and ARWU rankings, the picture was mixed. For example, the University of Auckland has remained relatively stable in both the ARWU and the THE rankings over time. While the University of Otago (and the University of Canterbury more recently) has been improving in the ARWU, Massey University and Victoria University of Wellington (VUW) have dropped in ranking in recent years. Both of the latter universities have also exhibited recent falls in THE ranking. We also compare the rankings of New Zealand universities with Australian universities. One New Zealand university, the University of Auckland, was placed among the Australian Group of Eight (G8) universities in all three rankings, while the University of Otago was placed just outside the G8, but above other Australian universities. The remaining listed New Zealand universities were generally spread among the remaining non-G8 universities. The performance of the Australian universities in the rankings, especially the non-G8 universities, suggests that wider trends are impacting on the Australasian universities. For example, all the listed Australasian universities dropped in the QS rankings between 2007 and 2013. The rise of universities from Asian countries in the rankings is one factor in displacing the Australasian universities

What young graduates earn when they leave study

This report examines outcomes for young people who complete a qualification in the New Zealand tertiary education system, looking at differences in incomes for different types of qualifications. Overview People take tertiary education for many reasons. They think about what they enjoy, what they are good at, what they are capable of and what will get them started on a career. Good careers are associated with better health, better well-being and more satisfying lives. So many young people are making their tertiary education choices to gain the skills they need for satisfying and rewarding work. They use a range of information sources to help them make these choices. The information in this report is designed to add to the data available to young people facing those decisions. This information is not just important to students and to their families. The Government makes a very large investment in tertiary education each year – funding tertiary education providers, providing subsidised student loans and granting student allowances. One major purpose of the Government’s investment is to help improve the New Zealand economy and society by raising the level of skill in the population – which helps make our society more productive, contributes to the creation of wealth and leads to better social outcomes. Studying the earnings of graduates is one way of looking at the contribution that the tertiary education system is making to New Zealand’s society and economy. So the information in this report contributes to an understanding of the value New Zealand receives for the investment we make in tertiary education. Key findings Earnings increase with the level of qualification completed. The biggest jump in earnings is between those with qualifications below degree level and those with degrees. Earnings remain consistently higher for those with higher qualifications. Those with higher qualifications consistently earn more for the first seven years post study, with no sign of these benefits decreasing. Employment rates increase with level of qualification gained. For example, in the first year after study, 54 percent of young bachelors graduates who stayed in New Zealand were in employment and 40 percent were in further study. Of young people who had completed a level 1-3 certificate and stayed in New Zealand, 35 percent were in employment and 48 percent were taking more study. Very few young people who complete a qualification at diploma level or above are on a benefit in the first seven years after study. For those who stay in New Zealand, the benefit rate is 6 percent for diploma graduates and 2 percent at bachelors level in each of the first seven years after study. But it is around 14 percent for those who graduated with certificates at levels 1-3. Earnings vary considerably by field of study. Young graduates with bachelors degrees in medicine earn the most of all bachelors graduates. The median income for medical graduates is over $110,300 five years after leaving study, compared to$51,600 for all young bachelors graduates. Bachelors degree graduates in creative arts have the lowest earnings among young bachelors graduates after five years and they have relatively high rates of benefit receipt. Some qualification types and some fields are associated with high rates of further study. Around half of all young people who complete a certificate or level 5-7 diploma move into further study the next year. Around 60 percent of young bachelors graduates in natural and physical sciences who stay in New Zealand were in further study one year after completion of a bachelors degree, and 32 percent after five years. Those who complete graduate certificates and diplomas have very high employment rates. Employment rates are around 80 percent or just below in the first three years after study for those who have completed a graduate certificate or diploma and who remain in New Zealand. Many of these graduates have completed this qualification as a way of improving their employment prospects or are studying while in employment. The effect of the recession on the earnings of young graduates is still apparent. Although the country as a whole has pulled out of recession, the effects on young people have lingered with graduate earnings continuing to drop in real terms compared to those reported in our first study, Moving on up, for most years after study and at almost all qualification levels. However, there are indications that the rate of decrease in earnings may have been slowing down for recent graduates by the end of the 2012 tax year

The measurement of the performance of New Zealand tertiary education institutions and the demand for their services

This thesis explored the measurement of performance of New Zealand tertiary education institutions (TEIs) and the demand for their services. This involved analysing the research performance of New Zealand universities, analysing the productive efficiency of New Zealand TEIs and examining the choice of provider by bachelor’s degree starters. Bibliometric data was used to measure the research productivity of New Zealand universities. This showed that following a fall during the early 2000s, the research productivity of New Zealand universities increased following the introduction of the Performance-Based Research Fund (PBRF). A multi-dimensional analysis of university research performance between 2000 and 2005 showed that no individual university was top in all four of the performance measures assessed. The overall performance of three universities, Massey University, Lincoln University and Auckland University of Technology, were noticeably below that of the other five universities. Data Envelopment Analysis (DEA) was then applied to input and output data of New Zealand TEIs to analyse their productive efficiency. In 2006, polytechnics that had: low levels of bachelor’s degree provision, were not regionally based, had a high proportion of subcontracting and were larger institutions, achieved higher levels of pure technical efficiency. The analysis showed that several polytechnics could improve their technical efficiency by reducing their scale of operations. In polytechnics, higher technical efficiency was associated with better financial performance. A number of technically efficient polytechnics struggled financially, indicating that the overall efficiency of the polytechnic sector was not high, or the funding model they operate under is not appropriate. The analysis also showed that decreasing bachelor’s degree provision, poor financial performance in the previous year, an increase in provision of community education, was associated with higher growth in total factor productivity between 1996 and 2006. The application of DEA to Australasian university data between 1997 and 2005 showed that New Zealand universities performed relatively well in terms of relative pure technical efficiency, compared with their Australian counterparts. However, the total factor productivity of New Zealand universities increased at a lower rate, on average, than that of the Australian Group of Eight and newer Australian universities. The application of DEA to a dataset of the participating TEIs in the PBRF showed that polytechnics had lower technical efficiency, on average, than other TEIs. The choices of bachelor’s degree starters in 2006 were analysed for evidence of a lack of parity of esteem between university and polytechnic degrees. The results showed that a lack of parity of esteem between polytechnic and university degrees may be influencing student choices. Students from higher deciles schools, with higher secondary school qualifications, Asians, students who travel for study, were all more likely to enrol in a university to start a bachelor’s degree. There was less clear cut evidence of a lack of parity of esteem between selected groupings of New Zealand universities. However, there did appear to be a lack of parity of esteem between the four older metropolitan universities and the two newest universities, with signs the former were held in higher esteem

The Use of Bibliometrics to Monitor the Performance of the New Zealand Tertiary Education System

This presentation discusses the role of bibliometrics in monitoring the performance of the New Zealand tertiary education system. The Ministry of Education uses aggregated data from Thomson Reuters to monitor the research performance of the tertiary sector and individual providers. The bibliometric data is used to benchmark the performance of New Zealand universities and is also used to monitor the impact of the Performance-Based Research Fund (PBRF) – the research funding allocation system for New Zealand’s tertiary education organisations. The bibliometric data shows that the median relative academic impact of New Zealand indexed research sits between Australian Go8 universities and non-Go8 universities. It also suggests there has been a shift in the research performance of New Zealand universities since the PBRF was introduced in 2004. Since then, the share of world indexed publications by New Zealand university authors has increased along with their share of world citations. The operation of the PBRF is constantly being reviewed and the Ministry of Education will watch the increased use of bibliometrics in the United Kingdom and Australia with interest

Exercise training for chronic heart failure (ExTraMATCH II): individual participant data meta-analysis of randomised controlled trials

This is the author accepted manuscript. The final version is available from NIHR Journals Library via the DOI in this record.Background: Current national and international guidelines on the management of heart failure (HF) recommend exercise-based cardiac rehabilitation (ExCR), but do not differentiate this recommendation according to patient subgroups. Objective(s): (1) to obtain definitive estimates of the impact of ExCR interventions versus control (no exercise intervention) on mortality, hospitalisation, exercise capacity, and health-related quality of life (HRQoL) in HF patients; (2) to determine the differential (subgroup) effects of ExCR in HF patients according to their age, gender, ejection fraction, aetiology, New York Heart Association (NYHA) class, and baseline exercise capacity; (3) to assess whether the change in exercise capacity mediates for the impact of the ExCR on final outcomes (mortality, hospitalisation, and HRQoL) and is an acceptable surrogate endpoint. Design: Individual participant data (IPD) meta-analysis Setting: An international literature review Participants: HF patients in randomised controlled trials (RCTs) of ExCR Interventions: ExCR for at least 3 weeks compared with no exercise control with 6 months follow-up Main outcome measures: mortality (all cause and HF-specific), hospitalisation (all-cause & HF-specific), exercise capacity, and HRQoL Data sources: Individual participant data from eligible RCTs 3 Review methods: RCTs from ExTraMATCH IPD meta-analysis and 2014 Cochrane systematic review of ExCR Results: Out of the 23 eligible RCTs (4,398 patients), 19 RCTs (3,990 patients) contributed data to this IPD meta-analysis. There was a wide variation in exercise programme prescriptions across included studies. Compared with control, there was no statistically significant difference in pooled time to event estimates in favour of ExCR although confidence intervals were wide: all-cause mortality: hazard ratio (HR) 0.83 (95% confidence interval (CI): 0.67 to 1.04), HF-related mortality: HR 0.84 (95% CI: 0.49 to 1.46), all-cause hospitalisation: HR 0.90 (95% CI: 0.76 to 1.06), and HF-related hospitalisation: HR 0.98 (95% CI: 0.72 to 1.35). There was a statistically significant difference in favour of ExCR for exercise capacity and HRQoL. Compared to control, at 12-months follow-up, improvements were seen in the six-minute walk test (6MWT) (mean: 21.0 metres, 95% CI: 1.57 to 40.4, and Minnesota Living with HF Questionnaire score (mean: -5.94, 95% CI: -1.0 to -10.9, lower scores indicate improved HRQoL). No strong evidence for differential intervention effects across patient characteristics was found for any outcomes. Moderate to good levels of correlation (R2 trial>50% & ρ>0.50) between peak oxygen uptake (VO2peak) or 6MWT with mortality and HRQoL were seen. Estimated surrogate threshold effect (STE) was an increase of 1.6 to 4.6 ml/kg/min for VO2peak. Limitations: Lack consistency in how included RCTs defined and collected the outcomes; we were unable to obtain IPD from all includable trials for all outcomes; and we did not seek patient level on exercise adherence. . Conclusions: In comparison to no exercise control, participation in ExCR improves the exercise and HRQoL in HF patients but appears to have no effect on their mortality or hospitalisation. No strong evidence was found of differential intervention effects of ExCR across patient characteristics. VO2peak and 6MWT may be suitable surrogate endpoints for the treatment effect of ExCR on mortality and HRQoL in HF.NIHR Health Technology Assessment programm

Spitzer Mid-Infrared Photometry of 500 - 750 K Brown Dwarfs

Mid-infrared data, including Spitzer warm-IRAC [3.6] and [4.5] photometry, is critical for understanding the cold population of brown dwarfs now being found, objects which have more in common with planets than stars. As effective temperature (T_eff) drops from 800 K to 400 K, the fraction of flux emitted beyond 3 microns increases rapidly, from about 40% to >75%. This rapid increase makes a color like H-[4.5] a very sensitive temperature indicator, and it can be combined with a gravity- and metallicity-sensitive color like H-K to constrain all three of these fundamental properties, which in turn gives us mass and age for these slowly cooling objects. Determination of mid-infrared color trends also allows better exploitation of the WISE mission by the community. We use new Spitzer Cycle 6 IRAC photometry, together with published data, to present trends of color with type for L0 to T10 dwarfs. We also use the atmospheric and evolutionary models of Saumon & Marley to investigate the masses and ages of 13 very late-type T dwarfs, which have H-[4.5] > 3.2 and T_eff ~ 500 K to 750 K.Comment: To be published in the on-line version of the Proceedings of Cool Stars 16 (ASP Conference Series). This is an updated version of Leggett et al. 2010 ApJ 710 1627; a photometry compilation is available at http://www.gemini.edu/staff/slegget

Validation of Exercise Capacity as a Surrogate Endpoint in Exercise-Based Rehabilitation for Heart Failure: A Meta-Analysis of Randomized Controlled Trials

This is the final version. Available on open access from Elsevier via the DOI in this recordObjectives: This study sought to validate exercise capacity (EC) as a surrogate for mortality, hospitalization, and health-related quality of life (HRQOL). Background: EC is often used as a primary outcome in exercise-based cardiac rehabilitation (CR) trials of heart failure (HF) via direct cardiorespiratory assessment of maximum oxygen uptake (VO2peak) or through submaximal tests, such as the 6-min walk test (6MWT). Methods: After a systematic review, 31 randomized trials of exercise-based CR compared with no exercise control (4,784 HF patients) were included. Outcomes were pooled using random effects meta-analyses, and inverse variance weighted linear regression equations were fitted to estimate the relationship between the CR on EC and all-cause mortality, hospitalization, and HRQOL. Spearman correlation coefficient (ρ), R2 at trial level, and surrogate threshold effect (STE) were calculated. STE represents the intercept of the prediction band of the regression line with null effect on the final outcome. Results: Exercise-based CR is associated with positive effects on EC measured through VO2peak (+3.10 ml/kg/min; 95% confidence interval [CI]: 2.01 to 4.20) or 6MWT (+41.15 m; 95% CI: 16.68 to 65.63) compared to control. The analyses showed a low level of association between improvements in EC (VO2peak or 6MWT) and mortality and hospitalization. Moderate levels of correlation between EC with HRQOL were seen (e.g., R2 <52%; |ρ| < 0.72). Estimated STE was an increase of 5 ml/kg/min for VO2peak and 80 m for 6MWT to predict a significant improvement in HRQOL. Conclusions: The study results indicate that EC is a poor surrogate endpoint for mortality and hospitalization but has moderate validity as a surrogate for HRQOL. Further research is needed to confirm these findings across other HF interventions.National Institute for Health Research (NIHR)University of Exete

Workforce predictive risk modelling: development of a model to identify general practices at risk of a supply−demand imbalance

Objective: This study aimed to develop a risk prediction model identifying general practices at risk of workforce supply–demand imbalance. Design: This is a secondary analysis of routine data on general practice workforce, patient experience and registered populations (2012 to 2016), combined with a census of general practitioners’ (GPs’) career intentions (2016). Setting/Participants: A hybrid approach was used to develop a model to predict workforce supply–demand imbalance based on practice factors using historical data (2012–2016) on all general practices in England (with over 1000 registered patients n=6398). The model was applied to current data (2016) to explore future risk for practices in South West England (n=368). Primary outcome measure: The primary outcome was a practice being in a state of workforce supply–demand imbalance operationally defined as being in the lowest third nationally of access scores according to the General Practice Patient Survey and the highest third nationally according to list size per full-time equivalent GP (weighted to the demographic distribution of registered patients and adjusted for deprivation). Results: Based on historical data, the predictive model had fair to good discriminatory ability to predict which practices faced supply–demand imbalance (area under receiver operating characteristic curve=0.755). Predictions using current data suggested that, on average, practices at highest risk of future supply–demand imbalance are currently characterised by having larger patient lists, employing more nurses, serving more deprived and younger populations, and having considerably worse patient experience ratings when compared with other practices. Incorporating findings from a survey of GP’s career intentions made little difference to predictions of future supply–demand risk status when compared with expected future workforce projections based only on routinely available data on GPs’ gender and age. Conclusions: It is possible to make reasonable predictions of an individual general practice’s future risk of undersupply of GP workforce with respect to its patient population. However, the predictions are inherently limited by the data available

Oesophageal varices, schistosomiasis, and mortality among patients admitted with haematemesis in Mwanza, Tanzania: a prospective cohort study.

BACKGROUND: Upper gastrointestinal bleeding (UGIB) is a common cause of hospital admissions worldwide. Aetiologies vary by sociodemographics and geography. Retrospective studies of endoscopies in much of Africa have documented oesophageal varices as a leading cause of UGIB. Prospective studies describing outcomes and associations with clinical factors are lacking. METHODS: We conducted a prospective cohort study at a referral hospital in Mwanza, Tanzania where schistosomiasis is endemic. Adults admitted with haematemesis underwent laboratory workup, schistosomiasis antigen testing and elective endoscopy, and were followed for two months for death or re-bleeding. We assessed predictors of endoscopic findings using logistic regression models, and determined prediction rules that maximised sensitivity and positive predictive value (PPV). RESULTS: Of 124 enrolled patients, 13 died within two months (10%); active schistosomiasis prevalence was 48%. 64/91(70%) patients had oesophageal varices. We found strong associations between varices and numerous demographic or clinical findings, permitting construction of simple, high-fidelity prediction rules for oesophageal varices applicable even in rural settings. Portal vein diameter ≥ 13 mm or water sourced from the lake yielded sensitivity, specificity, PPV and NPV >90% for oesophageal varices; presence of splenomegaly or water sourced from the lake maintained sensitivity and PPV >90%. CONCLUSIONS: Our results guide identification of patients, via ultrasound and clinical examination, likely to have varices for whom referral for endoscopy may be life-saving. Furthermore, they support empiric anti-schistosome treatment for patients with UGIB in schistosome-endemic regions. These interventions have potential to reduce UGIB-related morbidity and mortality in Africa